[Harp-L] SPAH 2010 Comb Test: Retraction & Apology

["Brendan Power" <bren@xxxxxxxxxxxxxxxxx>]

Thanks to Vern for his comprehensive round up of our recent Comb Test at
SPAH. 

It was interesting and enjoyable working together to make it happen. As he
says, it was an exercise in (trans-Atlantic) co-operation and compromise. In
the months leading up to the test there were lots of detailed, at times
impassioned emails flying back and forth, arguing for this procedure or
that. I wanted lots of combs, he only wanted two etc etc... (The compromise
there was that we had ten tests per player but only seven comb types, so we
each got what we wanted: the variety I desired and the duplication of tests
Vern insisted on). Similarly we eventually reached agreement on other
initially contentious issues.

We chose Marine Band reedplates and covers for every test, with just the
combs changing. In addition to the stock MB pear and bamboo combs and a
suitably sized ABS plastic comb, respected maker Chris Reynolds supplied us
with the others to the same specs, ones commonly used in custom harps:
brass, aluminium, corian, dymondwood.

Vern made a quick-change mechanism, which allowed for the combs to be
switched quickly, and had a large weight attached to mask the differing comb
weights (he has described the other ways in which comb differences were
masked in his report). On the day, Vern's task was to do the comb switches,
and mine to do the recordings. 

We recruited six excellent players to test the harp/comb assemblies: Dave
Barrett, Chris Michalek, Alex Paclin, Cara Cooke, Brandon Bailey and Jimi
Lee. We assured them that their test scores would remain confidential, and
this has been honoured. We also had the help of Winslow Yerxa and Michelle
LeFree, who were invaluable on the day in making the test go smoothly by
moving equipment around and making everyone comfortable.

The test went ahead; to our pleased surprise we got through it all in the
two hours allotted. Afterwards both Vern and I got busy with SPAH
activities, and it was only mid last week that I turned to the task of
getting the results and music clips online. That led to an emotional process
in which I've come full circle on my assessment of the test.

When I saw the score sheets, I was really surprised and very disappointed to
see that three testers had given uniform scores for all combs - two of them
from beginning to end and a third shortly after the start. After all our
hard work preparing the test, seeing a "3" in every box for every category
for every comb felt like a slap in the face, especially as the scores stayed
the same from start to finish. On the face of it, that looked like a
strategy that had been decided before the first test play. Unfortunately I
reacted impulsively, posting my angry feelings in my initial online Test
Report.

However, a week of reflection and private discussion has totally changed my
opinion. I now think that the uniform scores were the most significant and
useful data to come out of this test.

The main reason received for giving the uniform scores is that the test harp
assembly was insufficiently airtight to give a clear distinction between
tests. A reason given by one tester who marked uniform scores is that ".The
only perceived difference seemed to be in airtightness and responsiveness of
the reeds as a result of differences in screw tolerances and comb flatness
between each comb switch. the tests in the end were very inconclusive due to
large variances other than the comb materials". 

Another has written that "I didn't hear any difference between the
materials. What I did notice is the inconsistency between the poorly made
combs. Between them, it took more or less pressure to sound the notes. I
felt obvious air leaks and could tell Vern was inconsistent in the way he
screwed the plates to the comb... I thought the test was very poorly
executed and with sub par test objects."

That's a serious point, and deserves answering. As most of the testers are
used to playing high-level custom harps, possibly they were comparing the
test harp to the ones they normally play? However, it wasn't our goal to
replicate a custom harp, just to get average out-of-the-box playability
similar to a traditional 1896 Marine Band, while still allowing for quick
changes. I tried the test harp for the first time the day before the test,
and thought Vern had managed to achieve acceptable playability for test
purposes with his quick-change assembly mechanism. 

It sounded fine to me on test day when these six fine players blew it, and
their pieces sound good on the recorded clips. They are all great players,
and could make any harp sound good! However, on close listening you can hear
this was not a high-level harp, and the leakage referred to can be detected
in some places through the evidence of extra unintended notes sounding
occasionally.

Obviously some of the testers felt that this lack of perfect airtightness
was great enough to mask any comb effects there might (or might not) have
been. Essentially they thought the test harp was inadequate for the task, as
they have stated. Hence their uniform scores.

In our defence, we gave everyone the harp assembly to try in the half hour
before the test. It might have been preferable for those who felt the
assembly was inadequate to have simply withdrawn from the test at that
stage, rather than sat through two hours of their and everyone else's time
giving the same score for every test. We would have recorded their opinion,
but we could have given their seat to someone else who was happy to note
down their contrasting impressions of the different comb/harp assemblies, as
the three remaining testers were. 

But the three uniform testers decided to stay and sit it out, and I fully
accept their scoring was sincere. After a week of reflection I've gone from
being highly pissed off with them to thanking them, as I can now see how
useful and valid their contribution was. It makes me realise we set our
sights too low in accepting a stock nailed Marine Band as the benchmark for
our test harp, and should have aimed far higher.

The whole aim of our test was to record player perceptions of the possible
effects of differing comb materials. If the mating surfaces of the
reedplates and comb were not perfect, whatever effects there could be would
be minimised or masked entirely. This is what the uniform testers said, and
I entirely agree with them. While I don't accept the assertion that the
combs were not flat (they were checked against the light with a steel
ruler), it's undeniable that the overall assembly was not perfectly
airtight, as can be heard in the sound clips.

Though Vern may disagree, personally I have to admit that our test (while
well-intentioned and the product of lots of hard work from both of us),
missed the mark in its most critical element: the test harmonica. It did not
provide the required level of perfection in the seal between comb and
reedplates that was required to test the issue that we were aiming for. If a
comb is not seating to the reedplates perfectly then any possible effect
from the comb (if there) will be significantly reduced.

We have to thank the uniform scorers for being honest about that, and I
sincerely apologise for initially disparaging their scores. Instead of
testing an assembly at the level of a stock harp, to really look at this
question it should be done with a test harp that adheres to the level of a
top custom diatonic, with the superb level of airtightness between comb and
reedplates. As one of the scorers said "I would like to see results from
combs that are precisely flat and with plates that are properly set up."

I'd like to see that test happen too. It would take the close involvement of
a top custom harp maker to create the assembly and check it constantly
throughout the test, so there could be no quibbles from the test players. 

Though I now feel our test was fatally flawed and therefore inconclusive, I
believe Vern and I have contributed a lot in working out useful procedures
that can be used in such a future test. They are recorded in the
Instructions for Testers on the SPAH Test webpage.

I want to warmly thank everyone who took part in this test: Vern, Chris,
Brandon, Cara, Alex, Jimi, Dave, Winslow and Michelle. Despite the
difficulties and stresses, we had quite a few belly laughs in those two
hours, and invented a new scientific measuring gauge: The WINSPAN! Who
knows, it could be our most lasting legacy...

Brendan Power
WEBSITE: http://www.brendan-power.com
YOUTUBE: http://www.youtube.com/BrendanPowerMusic